

<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="zh" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="zh" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>Optuna: 一个超参数优化框架 &mdash; Optuna 1.4.0 文档</title>
  

  
  
    <link rel="shortcut icon" href="_static/favicon.ico"/>
  
  
  

  
  <script type="text/javascript" src="_static/js/modernizr.min.js"></script>
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
        <script src="_static/jquery.js"></script>
        <script src="_static/underscore.js"></script>
        <script src="_static/doctools.js"></script>
        <script src="_static/language_data.js"></script>
        <script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
    
    <script type="text/javascript" src="_static/js/theme.js"></script>

    

  
  <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
  <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="_static/css/custom.css" type="text/css" />
    <link rel="index" title="索引" href="genindex.html" />
    <link rel="search" title="搜索" href="search.html" />
    <link rel="next" title="安装" href="installation.html" /> 
</head>

<body class="wy-body-for-nav">

   
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search" >
          

          
            <a href="#" class="icon icon-home"> Optuna
          

          
          </a>

          
            
            
              <div class="version">
                1.4.0
              </div>
            
          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <p class="caption"><span class="caption-text">目录</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="installation.html">安装</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorial/index.html">教程</a></li>
<li class="toctree-l1"><a class="reference internal" href="reference/index.html">API Reference</a></li>
<li class="toctree-l1"><a class="reference internal" href="faq.html">常见问题</a></li>
</ul>

            
          
        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="#">Optuna</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="#">Docs</a> &raquo;</li>
        
      <li>Optuna: 一个超参数优化框架</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
            
            <a href="_sources/index.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <p><a class="reference internal" href="https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png"><img alt="OPTUNA" src="https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png" style="width: 800px;" /></a></p>
<div class="section" id="optuna-a-hyperparameter-optimization-framework">
<h1>Optuna: 一个超参数优化框架<a class="headerlink" href="#optuna-a-hyperparameter-optimization-framework" title="永久链接至标题">¶</a></h1>
<p><em>Optuna</em> 是一个特别为机器学习设计的自动超参数优化软件框架。它具有命令式的，<em>define-by-run</em> 风格的 API。由于这种 API 的存在，用 Optuna 编写的代码模块化程度很高，Optuna 的用户因此也可以动态地构造超参数的搜索空间。</p>
<div class="section" id="key-features">
<h2>主要特点<a class="headerlink" href="#key-features" title="永久链接至标题">¶</a></h2>
<p>Optuna 有如下现代化的功能：</p>
<ul class="simple">
<li><p><a class="reference internal" href="tutorial/first.html"><span class="doc">轻量级、多功能和跨平台架构</span></a></p></li>
<li><p><a class="reference internal" href="tutorial/distributed.html"><span class="doc">并行的分布式优化</span></a></p></li>
<li><p><a class="reference internal" href="tutorial/pruning.html"><span class="doc">对不理想实验 (trial) 的剪枝 (pruning)</span></a></p></li>
</ul>
</div>
<div class="section" id="basic-concepts">
<h2>基本概念<a class="headerlink" href="#basic-concepts" title="永久链接至标题">¶</a></h2>
<p>我们以如下方式使用 <em>study</em> 和 <em>trial</em> 这两个术语：</p>
<ul class="simple">
<li><p>Study: 基于目标函数的优化过程</p></li>
<li><p>Trial: 目标函数的单次执行过程</p></li>
</ul>
<p>请参考下面的示例代码。一个 <em>study</em> 的目的是通过多次 <em>trial</em> (例如 <code class="docutils literal notranslate"><span class="pre">n_trials=100</span></code> ) 来找出最佳的超参数值集（比如选择 <code class="docutils literal notranslate"><span class="pre">classifier</span></code> 还是 <code class="docutils literal notranslate"><span class="pre">svm_c</span></code>）。而 Optuna 旨在加速和自动化此类 <em>study</em> 优化过程。</p>
<p><a class="reference external" href="http://colab.research.google.com/github/optuna/optuna/blob/master/examples/quickstart.ipynb"><img alt="Open in Colab" src="https://colab.research.google.com/assets/colab-badge.svg" /></a></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">...</span>

<span class="c1"># Define an objective function to be minimized.</span>
<span class="k">def</span> <span class="nf">objective</span><span class="p">(</span><span class="n">trial</span><span class="p">):</span>

    <span class="c1"># Invoke suggest methods of a Trial object to generate hyperparameters.</span>
    <span class="n">regressor_name</span> <span class="o">=</span> <span class="n">trial</span><span class="o">.</span><span class="n">suggest_categorical</span><span class="p">(</span><span class="s1">&#39;classifier&#39;</span><span class="p">,</span> <span class="p">[</span><span class="s1">&#39;SVR&#39;</span><span class="p">,</span> <span class="s1">&#39;RandomForest&#39;</span><span class="p">])</span>
    <span class="k">if</span> <span class="n">regressor_name</span> <span class="o">==</span> <span class="s1">&#39;SVR&#39;</span><span class="p">:</span>
        <span class="n">svr_c</span> <span class="o">=</span> <span class="n">trial</span><span class="o">.</span><span class="n">suggest_loguniform</span><span class="p">(</span><span class="s1">&#39;svr_c&#39;</span><span class="p">,</span> <span class="mf">1e-10</span><span class="p">,</span> <span class="mf">1e10</span><span class="p">)</span>
        <span class="n">regressor_obj</span> <span class="o">=</span> <span class="n">sklearn</span><span class="o">.</span><span class="n">svm</span><span class="o">.</span><span class="n">SVR</span><span class="p">(</span><span class="n">C</span><span class="o">=</span><span class="n">svr_c</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">rf_max_depth</span> <span class="o">=</span> <span class="n">trial</span><span class="o">.</span><span class="n">suggest_int</span><span class="p">(</span><span class="s1">&#39;rf_max_depth&#39;</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">32</span><span class="p">)</span>
        <span class="n">regressor_obj</span> <span class="o">=</span> <span class="n">sklearn</span><span class="o">.</span><span class="n">ensemble</span><span class="o">.</span><span class="n">RandomForestRegressor</span><span class="p">(</span><span class="n">max_depth</span><span class="o">=</span><span class="n">rf_max_depth</span><span class="p">)</span>

    <span class="n">X</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">sklearn</span><span class="o">.</span><span class="n">datasets</span><span class="o">.</span><span class="n">load_boston</span><span class="p">(</span><span class="n">return_X_y</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
    <span class="n">X_train</span><span class="p">,</span> <span class="n">X_val</span><span class="p">,</span> <span class="n">y_train</span><span class="p">,</span> <span class="n">y_val</span> <span class="o">=</span> <span class="n">sklearn</span><span class="o">.</span><span class="n">model_selection</span><span class="o">.</span><span class="n">train_test_split</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">random_state</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>

    <span class="n">regressor_obj</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">y_train</span><span class="p">)</span>
    <span class="n">y_pred</span> <span class="o">=</span> <span class="n">regressor_obj</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_val</span><span class="p">)</span>

    <span class="n">error</span> <span class="o">=</span> <span class="n">sklearn</span><span class="o">.</span><span class="n">metrics</span><span class="o">.</span><span class="n">mean_squared_error</span><span class="p">(</span><span class="n">y_val</span><span class="p">,</span> <span class="n">y_pred</span><span class="p">)</span>

    <span class="k">return</span> <span class="n">error</span>  <span class="c1"># An objective value linked with the Trial object.</span>

<span class="n">study</span> <span class="o">=</span> <span class="n">optuna</span><span class="o">.</span><span class="n">create_study</span><span class="p">()</span>  <span class="c1"># Create a new study.</span>
<span class="n">study</span><span class="o">.</span><span class="n">optimize</span><span class="p">(</span><span class="n">objective</span><span class="p">,</span> <span class="n">n_trials</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span>  <span class="c1"># Invoke optimization of the objective function.</span>
</pre></div>
</div>
</div>
<div class="section" id="communication">
<h2>Communication<a class="headerlink" href="#communication" title="永久链接至标题">¶</a></h2>
<ul class="simple">
<li><p>可在 <a class="reference external" href="https://github.com/optuna/optuna/issues">GitHub Issues</a> 报告bug、提feature request 和问问题。</p></li>
<li><p>可在 <a class="reference external" href="https://gitter.im/optuna/optuna">Gitter</a> 与开发者互动.</p></li>
<li><p>可在 <a class="reference external" href="https://stackoverflow.com/questions/tagged/optuna">StackOverflow</a> 提问。</p></li>
</ul>
</div>
<div class="section" id="contribution">
<h2>贡献<a class="headerlink" href="#contribution" title="永久链接至标题">¶</a></h2>
<p>欢迎大家对Optuna的一切贡献！但是在发送pull request时请遵从 <a class="reference external" href="https://github.com/optuna/optuna/blob/master/CONTRIBUTING.md">contribution guide</a> 的规范.</p>
</div>
<div class="section" id="license">
<h2>License<a class="headerlink" href="#license" title="永久链接至标题">¶</a></h2>
<p>MIT License (see <a class="reference external" href="https://github.com/optuna/optuna/blob/master/LICENSE">LICENSE</a>).</p>
</div>
<div class="section" id="reference">
<h2>Reference<a class="headerlink" href="#reference" title="永久链接至标题">¶</a></h2>
<p>Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori
Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization
Framework. In KDD (<a class="reference external" href="https://arxiv.org/abs/1907.10902">arXiv</a>).</p>
<div class="toctree-wrapper compound">
<p class="caption"><span class="caption-text">目录</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="installation.html">安装</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorial/index.html">教程</a><ul>
<li class="toctree-l2"><a class="reference internal" href="tutorial/first.html">第一个优化例子</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial/configurations.html">高级配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial/rdb.html">用 RDB 后端保存/恢复 Study</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial/distributed.html">分布式优化</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial/cli.html">命令行界面</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial/attributes.html">用户定义属性</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial/pruning.html">对无望的 Trial 进行剪枝 (Pruning)</a></li>
<li class="toctree-l2"><a class="reference internal" href="tutorial/sampler.html">用户定义的采样器 (Sampler)</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="reference/index.html">API Reference</a><ul>
<li class="toctree-l2"><a class="reference internal" href="reference/core.html">Core</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/cli.html">Command Line Interface</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/distributions.html">Distributions</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/exceptions.html">Exceptions</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/importance.html">Hyperparameter Importance</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/integration.html">Integration</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/logging.html">Logging</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/multi_objective/index.html">Multi-objective</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/pruners.html">Pruners</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/samplers.html">Samplers</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/storages.html">Storages</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/structs.html">Structs</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/study.html">Study</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/trial.html">Trial</a></li>
<li class="toctree-l2"><a class="reference internal" href="reference/visualization.html">Visualization</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="faq.html">常见问题</a><ul>
<li class="toctree-l2"><a class="reference internal" href="faq.html#can-i-use-optuna-with-x-where-x-is-your-favorite-ml-library">某某库可以和 Optuna 配合使用吗？（某某是你常用的机器学习库）</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-to-define-objective-functions-that-have-own-arguments">如何定义带有额外参数的目标函数？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#can-i-use-optuna-without-remote-rdb-servers">没有远程 RDB 的情况下可以使用 Optuna 吗？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-can-i-save-and-resume-studies">如何保存和恢复 study？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-to-suppress-log-messages-of-optuna">如何禁用 Optuna 的日志信息？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-to-save-machine-learning-models-trained-in-objective-functions">如何在目标函数中保存训练好的机器学习模型？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-can-i-obtain-reproducible-optimization-results">如何获得可复现的优化结果？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-are-exceptions-from-trials-handled">Trial 是如何处理抛出异常的？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-are-nans-returned-by-trials-handled">Trial 返回的 NaN 是如何处理的？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#what-happens-when-i-dynamically-alter-a-search-space">动态地改变搜索空间会导致怎样的结果？</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-can-i-use-two-gpus-for-evaluating-two-trials-simultaneously">如何在两块 GPU 上同时跑两个 trial?</a></li>
<li class="toctree-l2"><a class="reference internal" href="faq.html#how-can-i-test-my-objective-functions">如何对目标函数进行测试？</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div>
<div class="section" id="indices-and-tables">
<h1>Indices and tables<a class="headerlink" href="#indices-and-tables" title="永久链接至标题">¶</a></h1>
<ul class="simple">
<li><p><a class="reference internal" href="genindex.html"><span class="std std-ref">索引</span></a></p></li>
<li><p><a class="reference internal" href="py-modindex.html"><span class="std std-ref">模块索引</span></a></p></li>
<li><p><a class="reference internal" href="search.html"><span class="std std-ref">搜索页面</span></a></p></li>
</ul>
</div>


           </div>
           
          </div>
          <footer>
  
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
      
        <a href="installation.html" class="btn btn-neutral float-right" title="安装" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
      
      
    </div>
  

  <hr/>

  <div role="contentinfo">
    <p>
        &copy; Copyright 2018, Optuna Contributors.

    </p>
  </div>
  Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
    <a href="privacy.html">Privacy Policy</a>.
     


</footer>

        </div>
      </div>

    </section>

  </div>
  


  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
    <!-- Theme Analytics -->
    <script>
    (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
      (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
      m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
    })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

    ga('create', 'UA-55135190-8', 'auto');
    ga('send', 'pageview');
    </script>

    
   

</body>
</html>